Method for dynamically managing content delivery

ABSTRACT

Methods and systems are provided for bitrate adaptation of a video asset to be streamed to a client device for playback. The method includes selecting a representation from a manifest which expresses a set of representations available for each chunk of the video asset and generating a dynamic manifest for the video asset in which the representation selected for the at least one chunk is recommended for streaming to the client device. The selection of the representation recommended for the chunk may be based on at least one of historic viewing behavior of previous viewers of the chunk, content analysis information for the chunk, a level of available network bandwidth, a level of available network storage, and data rate utilization information of network resources including current, average, peak, and minimum data rate of network resources.

BACKGROUND

Network digital video recorders (nDVR), network personal video recorders(nPVR), remote storage digital video recorder (rs-DVR), and likeequipment are network-based digital video recorders that may be storedor located on the cloud at a server location or at a content provider'slocation rather than at a consumer's private location or home. Suchnetwork devices have effectively increased the consumer's ability totime shift the consumption of programs (i.e., to record, download, orstream a program and ultimately consume the program or parts thereof ata later time that best suits the consumer). This ability to time shiftalso provides the consumer with enhanced power to consume only selectedportions of programs by, for example, skipping or fast-forwardingthrough portions of recorded content, and also to consume parts of aprogram multiple times via use of rewinding or the like.

In an nDVR or time-shifted content delivery system, video contentavailable for playback may be recorded, transcoded, and stored inseveral video formats. Typically, each format consists of a differentvideo resolution and bitrate, to enable adaptive bitrate streaming. Themultiplicity of different stream formats and bit rates enables thecontent to be sourced to devices with different capabilities, such ashigh definition televisions of wide ranging sizes, personal computers,tablet computers, smart phones, and other client devices. In addition,the different bit rates support adaptive streaming, whereby thereceiving client has the ability to measure network congestion andrequest a lower or higher bit rate stream from the source which mayeliminate visual impairments caused by network congestion (e.g.macro-blocking due to dropped packets, frozen frames) at the expense ofhigher resolution video. Any of several video delivery protocols, suchas, for instance, HTTP Live Streaming, may be used to deliver theadaptive bitrate content to end users.

SUMMARY

A method of bitrate adaptation for a video asset to be streamed to aclient device for playback is provided. The video asset is deliverablein a sequence of chunks available in a plurality of differentrepresentations with each representation providing a differentpre-determined level of video quality due to different bitrate or videoresolution encoding. The method includes a step of selecting arepresentation from the plurality of different representations for atleast one of the chunks of the video asset from a manifest whichexpresses a set of representations available for each of the chunks ofthe video asset. The method also includes the step of generating adynamic manifest for the video asset in which the representationselected during the selecting step for the chunk is recommended forstreaming to the client device. During the selecting step, a selectionof the representation recommended for the chunk is based on at least oneof data of historic viewing behavior of previous viewers of the chunk ofthe video asset and data of content analysis information for the chunkof the video asset. The data of historic viewing behavior provides anindication of a level of historic viewer interest of each the chunks ofthe video asset and the data of content analysis information provides anindication of a level of at least one of importance, relevance, andactivity of each of the chunks of the video asset.

A second embodiment of a method of bitrate adaptation for a video assetto be streamed to a client device for playback includes a step ofselecting a representation from a plurality of different representationsfor at least one chunk of a video asset from a manifest and a step ofgenerating a dynamic manifest for the video asset in which therepresentation selected during the selecting step for the chunk isrecommended for streaming to the client device. During the selectingstep, a selection of the representation recommended for the chunk isbased on at least one of a level of available network bandwidth, a levelof available network storage, and data rate utilization information ofnetwork resources including at least one of current, average, peak, andminimum data rate of network resources.

A system of bitrate adaptation for a video asset to be streamed to aclient device for playback is also provided. The system compriseselectronic apparatus having at least one processor configured to selecta representation from a plurality of different representations for atleast one chunk of a video asset from an existing manifest whichexpresses a set of representations available for each chunk of the videoasset and generate a dynamic manifest for the video asset in which therepresentation selected for the chunk is recommended for streaming tothe client device.

A second embodiment of a system of bitrate adaptation for a video assetto be streamed to a client device for playback includes a deliverynetwork server configured to receive data of historic user behaviorcorresponding to a video asset, data of content event informationderived from media analysis of the video asset, and a manifest for thevideo asset that expresses a set of representations available for eachchunk of the video asset and to generate activity information ofhistoric user behavior and content event information for each of thechunks of the video asset. The system also includes a local networkgateway device that characterizes local network activity and anindication of which manifest bit-rate ranges are best suited for a localnetwork environment. The local network gateway device is configured toreceive the activity information from the delivery network server andselect representations for the chunks of the video asset to be streamedbased on the activity information and local network cache and bandwidthusage.

BRIEF DESCRIPTION OF THE DRAWINGS

Various features of the embodiments described in the following detaileddescription can be more fully appreciated when considered with referenceto the accompanying figures, wherein the same numbers refer to the sameelements.

FIG. 1 is a graph showing a set of fixed manifests for a video asset inwhich a constant bitrate selection is provided for all chunks of thevideo asset according to the prior art.

FIG. 2 is a graph showing a set of dynamic manifests for a video assetin which bitrate selections vary form chunk-to-chunk based ondifferences in historic user behavior and content analysis informationfor each chunk in accordance to an embodiment.

FIG. 3 is a schematic view of a system for generating and delivering adynamic manifest file to client devices in accordance to an embodiment.

FIG. 4 is a schematic view of a packager and dynamic manifest mapper inaccordance to an embodiment.

FIG. 5 is a schematic view of a packager, dynamic manipulator mapper,and network-aware manifest manager in accordance to an embodiment.

FIG. 6 is a schematic view of a fusion service server, packager, andhome gateway in accordance to an embodiment.

FIG. 7 is a flow diagram of a method of bitrate adaptation in accordanceto an embodiment.

FIG. 8 is a flow diagram of a method of bitrate adaptation in accordanceto an embodiment.

DETAILED DESCRIPTION

For simplicity and illustrative purposes, the principles of theembodiments are described by referring mainly to examples thereof. Inthe following description, numerous specific details are set forth inorder to provide a thorough understanding of the embodiments. It will beapparent however, to one of ordinary skill in the art, that theembodiments may be practiced without limitation to these specificdetails. In some instances, well known methods and structures have notbeen described in detail so as not to unnecessarily obscure theembodiments.

As discussed above, bitrate adaptation (i.e., the selection of thestream to receive and present) has conventionally been performed at theclient, and has been determined based solely on download speed andclient resources. As a result, bitrate decisions may result in eitherpoor video presentation or (conversely) unnecessarily wasted bandwidthand may not directly account for network resource loads. Thus,conventional bitrate adaption techniques may lead to poor performance atthe client side and an increased slowness on the network side.

Thus, according to some embodiments disclosed herein, a method andsystem are provided that modifies the conventional bitrate adaptationdecision to optimize presentation and bandwidth usage.

In addition, according to other embodiments disclosed herein, a methodand system are provided that proactively manage a network load acrossall client sessions by considering a current state of the deliverynetwork when adapting bitrates sent to clients served by networkresources.

Accordingly, these methods provide a proactive system that may determinerelative importance of content segments from historic user interactionsor content information for purposes of optimizing adaptive bitstreamdelivery. Furthermore, this optimization may be combined with networkactivity information to provide a more proactive adjustment to the loadof the delivery network.

Dynamic Manifest Mapping Optimizing QoS for nDVR Content Delivery

According to a first embodiment, information about the content, whichcan be used to influence a bitrate adaptation decision, may includehistoric viewing behavior data and content media analysis data.

With respect to viewing behavior data, such data may be obtained throughcollection of user interactions while watching content, such as fastforward, rewind, pause, play, stop, skip ahead, skip back/replay, slowmotion, and other similar commands that affect the playback of thevideo. Often, these interactions occur via a remote control whilewatching the content on a primary (TV) screen, but could also occur on asecondary (phone/tablet) device, in a web browser on a computer, or anyother manner in which video can be delivered. Interactions for manyviewers can be collected and analyzed. When such historic information isviewed in aggregate, it may indicate portions of content which may havebeen of lower interest (many fast forward or skip ahead events) or ofhigher interest (many rewinds or replay events) during past viewings ofthe content.

With respect to content media analysis data, such data may be obtainedthrough automated analysis of video, audio and text components of avideo asset. This analysis can determine several features of the videoasset, including scene changes, identification of advertisements, levelof activity or motion, relative importance or relevance of segments, andsimilar information.

According to an embodiment, a so-called Dynamic Manifest Manipulator orMapper is configured to receive, as an input, user behavior related to acontent stream or video asset and/or input content event stream(temporal metadata) derived from media analysis of the content or videoasset. Further, the Dynamic Manifest Manipulator is configured toreceive input of a manifest file for the content or video asset andcreate a new manifest file (also referred to as a “dynamic manifestfile”) based on the above referenced inputs. Accordingly, knowledge ofhistoric or past user behavior with respect to viewing the content orvideo asset permits a more efficient and effective manifest file to becreated for delivery of the content or video asset based on semantic anduser value.

For purposes of example, a video asset (A_(i)) is referenced by a timeparameter (t) with values ranging from [0,T_(i)]. The viewing behaviordata (V_(i)) for the video asset (A_(i)) can be expressed for any pointin time (t) of the video asset. Thus, by definition, the value ofV_(i)(t) indicates a level of interest or importance of a section ofcontent (for instance, a higher value may correspond to a higher viewerinterest level and a lower value may correspond to a lower viewerinterest level).

In a similar manner, a measure of importance or relevance from mediaanalysis data (E_(i)) for the video asset (A_(i)) can be expressed forany time (t) within the video asset, for instance, with higher values ofE_(i)(t) corresponding to higher levels of importance, relevance, oractivity. Thus, for each time parameter in a video asset, a relativelevel of viewing behavior and a relative level of importance, relevance,or activity may be ascertained.

As discussed above with respect to adaptive streaming, each video asset(A_(i)) may have several representations, each at a different videobitrate and/or resolution. As an example, each video asset (A_(i)) mayhave “n” different representations, each representation (j) having anaffiliated bitrate: b_(j), j∈[1, n]. The bitrates are ordered b₁< . . .<b_(n) and lower bitrates generally correspond to lower fidelity videoquality due to the lower video encoding bitrate and/or lower videoresolution. Conversely, higher bitrates generally correspond to higherfidelity video quality due to higher video encoding bitrate and/orhigher video resolution

A manifest (M_(i)) expresses for asset (A_(i)) the set of bitraterepresentations available for the asset (A_(i)), and, for a time (t),indicates the content chunks available for playback in each of thebitrate representations. In adaptive bitrate streaming, the video asset(A_(i)) is chunked into discrete time portions, which potentiallypermits the playback of each chunk at a different bitrate. As anexample, each chunk may correspond to 2 to 10 seconds of video duringplayback; of course, other chunk lengths are also possible. Thus, theasset (A_(i)) may be indexed by integer chunk numbers (c, c∈[0, C_(i)]).Effectively, a surjective time mapping function may map each time valueonto a corresponding chunk (Θ(t₀)=c₀).

According to an embodiment, an extension referred to as a dynamicmanifest (DM_(i)) may be generated in addition to the typical manifest(M_(i)) for asset (A_(i)). The dynamic manifest (DM_(i)) indicates anindividually recommended bitrate stream (j) for each chunk (c) asopposed to the use of the same bitrate representation for all chunks ofa video asset. Thus, different bitrate representations may berecommended for different chunks comprising a single video asset Therecommended stream by the dynamic manifest (DM_(i)) is determined basedon the content information, i.e., viewing behavior data (V_(i)) andmedia analysis data (E_(i)), for the time period affiliated with theparticular chunk (c).

According to an embodiment, discrete content information functions maybe produced. Thus, for example, a continuous time function (V_(i)(t))corresponds to a discrete time function (V_(i)(c)). Thus, for example, atime range t∈[t₁,t₂] may map onto a single content chunk (c), i.e.,∀t∈[t₁,t₂], (Θ(t)=c). The value of V_(i)(t), t∈[t₁,t₂] may be averagedand mapped onto one of the discrete bitrate values [1, n]. A similarprocedure may be performed for the creation of E_(i)(c) from E_(i)(t).

Accordingly, for each chunk (c) in an asset (A_(i)), functions V_(i)(c)and E_(i)(c) are assigned a discrete bitrate value [1, n]. A relativelylower value (e.g., 1) may correspond to viewing behavior data andcontent media analysis data that indicate, respectively, that the userengagement with this particular content portion and the importance orrelevance of this particular content portion is relatively low. As aresult, a lower bitrate may be appropriate for this chunk (c). Theconverse may be true for higher values (e.g., n).

The values of functions V_(i)(c) and E_(i)(c) may be combined in anysuitable manner to determine the dynamic manifest recommended bitrate(DM_(i)) of asset (A_(i)). As an example of one embodiment, thecombining function may be that of the maximum: DM_(i)(c)=max(V_(i)(c),E_(i)(c)). Other examples include embodiments that employ the minimumvalue (min(V_(i)(c), E_(i)(c))) or that average the values(avg(V_(i)(c), E_(i)(c))) for providing a combining function.

Accordingly to some embodiments, the rate of change of the bitrate fromone chunk to the next may be limited on the basis that a significantbitrate change may be visually jarring during playback. For example, thechange rate between adjacent chunks of video may be limited to one stepin bitrate per chunk. Thus, a rate limited dynamic manifest recommendedbitrate (RLDM_(i)(c)) for chunk number (c) may be expressed, as follows:

if DM_(i)(c)−RLDM_(i)(c−1)>1, then RLDM_(i)(c)=RLDM_(i)(c−1)+1; or

if DM_(i)(c)−RLDM_(i)(c−1)<1, then RLDM_(i)(c)=RLDM_(i)(c−1)−1; or

otherwise RLDM_(i)(c)=DM_(i)(c).

Further, the range of bitrates for certain portions of the video assetmay be restricted, such as to guarantee a minimum bitrate foradvertising. In particular, the content media analysis data can produceadditional information about the content at a particular point in time;this information may indicate an advertisement, for example. Theoperator of a video delivery service may provide guarantees about thequality of service (QoS) of advertising being presented. Thus, when achunk (c) contains an advertisement, the dynamic manifest recommendedbitrate may be modified so that it is larger than some predeterminedlimiting value (e.g., limit=2). In such a case, for example, DM_(i)(c)may be set to equal max(V_(i)(c), E_(i)(c), limit).

For purpose of communicating dynamic manifest information to clients,the dynamic manifest recommended bitrate is determined for each chunk ofthe video asset and communicated to the client, that is, the device thatis receiving the video chunks and playing the video for the user.According to one embodiment, the DM_(i) or RLDM_(i) values may becarried in a Master Playlist. If HTTP Live Streaming is the protocolused for delivery, the dynamic manifest information may be carried, forinstance, in the EXT-X-SESSION-DATA tag. The EXT-X-SESSION-DATA tagpermits arbitrary session data to be carried in a Master Playlist. Inparticular, the DATA-ID field of this tag may be set to indicate thatthe data describes dynamic manifest recommended bitrate information, andthe VALUE field of this tag may contain, for example, the chunk numberand the recommended bitrate value for several chunks. This informationcan be encoded in JavaScript Object Notation (JSON) format, for example,or any suitable serialization format. For example, see the followingformat: [{chunk: 3401, bitrate: 3}, {chunk: 3402, bitrate: 3}, {chunk:3403, bitrate: 2}]. The EXT-X-SESSION-DATA tag may be updatedcontinuously or continually in the Master Playlist as new chunks of thevideo asset become available.

Accordingly, the client can receive the EXT-X-SESSION-DATA when itdownloads a playlist, and the client may follow the bitrate suggestionsexplicitly, downloading the recommended bitrate for each chunk.Alternatively, the client may use the recommended bitrate information toalter the bitrate choice for a chunk, without following therecommendation exactly. By way of example, if a client's algorithm(based on download speed and client performance) indicates that abitrate “4” of a certain chunk is to be used, while the dynamic manifestrecommended bitrate indicates a bitrate “2”, then the client may averagethese decisions and select a bitrate “3” for the chunk.

In an alternative embodiment, the dynamic manifest information may notbe carried explicitly in the Master Playlist. For instance, thisembodiment may be useful if the client does not implement the behaviorto use the recommended bitrate information. In HTTP Live Streaming, aMaster Playlist contains an entry for each available bitrate, with eachentry indicating a bandwidth (bitrate) value, and a link to the mediaplaylist corresponding to the bitrate. Further, a media playlist foreach bitrate is also created and updated. In this embodiment, dynamicmanifest information may be used to create an additional media playlist,so that if there are (n) available bitrates, then there will be n+1media playlists. Normally, a media playlist contains only chunks of thevideo asset for a single bitrate. The additional media playlist, whichmay be called a dynamic manifest media playlist, may contain a link toeach chunk corresponding to the recommended bitrate. Thus, a client mayrealize the recommendation of the dynamic manifest simply by playing thechunks indicated in the dynamic manifest media playlist.

Further embodiments may create multiple dynamic manifest mediaplaylists. For instance, a primary dynamic manifest media playlist maybe defined as the playlist containing chunks according to DM_(i)(c)discussed above. Then, a lower bitrate dynamic manifest media playlistmat be created to contain chunks according to max(DM_(i)(c)−p, 1),where, for example, p=1. Similarly, a higher bitrate dynamic manifestmedia playlist may be created and contain chunks according tomin(DM_(i)(c)+p, n). In other words, these provide alternate versions ofthe dynamic manifest recommendations by shifting all recommendedbitrates up or down by a fixed number of steps. Using this procedure, aset of dynamic manifests can be created for use by the client.

A set of dynamic manifests may be provided to a client for the purposesof achieving several advantages. First, the client implementation andbehavior need not be aware of the dynamic manifest operation; thus, thistechnique may be used with legacy clients. Second, it offers the clientthe ability to change the bitrate of the stream being played in responseto network speed and other client performance factors. Third, it mayoffer the benefits of the dynamic manifest recommendation, namely, tocause the client to download and play lower or higher bitrate chunks inresponse to the characteristics of the video itself.

To illustrate the use of dynamic manifests, consider the example titled“Fixed Manifests” 10 shown in FIG. 1, which represents the conventionaloperation of a protocol such as HTTP Live Streaming. In FIG. 1, thereare n=5 bitrates offered for the video asset. As a result, there arefive media playlists listed within the master playlist, one for eachbitrate. Each media playlist and the chunks it contains are representedby one of the straight horizontal lines in the graph, 12, 14, 16, 18 and20. Thus, for a bitrate selection of “3”, the middle horizontal line 16in FIG. 1 represents a media playlist where each and every chunk of thevideo asset contains video encoded at only bitrate “3”. The client canchoose to play chunks from any playlist as it plays back the content.

In contrast, FIG. 2 depicts an example titled “Dynamic Manifests” 22according to an embodiment. In this example, there are three dynamicmanifest media playlists offered for the video asset, each representedby a different relatively jagged line, 24, 26, and 28, in the graph.Similar to the previous Example, there are n=5 bitrates. Line 26represents the primary dynamic manifest media playlist, the upper line28 represents a higher bitrate dynamic manifest media playlist, and thelower line 24 represents a lower bitrate dynamic manifest mediaplaylist. As shown in FIG. 2, line 26 may be located between lines 24and 28 or, for some chunks, line 26 may overlap or be the same as line24 or 28. If the client selects only the primary playlist 26, it willreceive chunks at various bitrates (between 1 and 5) as it plays thecontent. However, the client has the flexibility to change to a lower orhigher bitrate playlist, 24 or 28, depending on client conditions.Playlist 24 includes chunks at various bitrates between 1 and 4, andplaylist 28 includes various bitrates between 2 and 5.

A system 30 for dynamic manifest mapping that optimizes Quality ofService (QoS) for content delivery as discussed above is shown in FIG.3. The nDVR 32 may be located in the cloud or like location and contentor video assets may be delivered therefrom via a content deliverynetwork (CDN) 34 to client devices (i.e., set top box, computer, tabletcomputer, smart phone, etc.) 36 located at the consumer location. ThenDVR 32 may be configured with an Analytics system 38, electronicprogram guide metadata 40, subscriber account information 42, ascheduler 44, and a fulfillment manager 46. Video streams from a sourcemay be received by a multiplexor 48 and may be encoded by a transcoder50. Linear video streams may be fed directly from the transcoder 50 tothe CDN 34 and ultimately to the client devices 36. The transcoder 50may also provide multi-bitrate (MBR) transport streams (TS) to recorders52 communicating with a recorder control module (RCM) 56. The RCM 56communicates with the nDVR 32 via a recorder (RM) 58, and the recorders52 communicate with the nDVR 32 via media analysis framework (MAF)equipment 60.

The recorders 52 also communicating with a packager 54 having a contentdelivery controller (CDC) 62 and a manifest delivery controller (MDC) 64which in turn communicates with the CDN 34 and client devices 36.Finally, for providing the dynamic manifest discussed above, a DynamicManifest Mapper (DMM) 66 communicates with the nDVR 32 and the MDC 64.

The above arrangement permits analytics to be collected from informationof users' interactions with content (linear and/or recorded), forinstance, FF, REW, PAUSE, and the like. In addition, recorded contentmay be analyzed to create activity stream (low-level & semantic) that isinput into the analytics system 38. The DMM 66 creates a reference mapand sequence of content chunks that is usage and content aware either ata subscriber or Service Group level as described above.

As best shown in FIG. 4, the DMM 66 receives as inputs from theanalytics system 38 of the nDVR 32: V_(i)(t) (i.e., user viewingbehavior stream for video asset A_(i)); and E_(i)(t) (i.e., mediaanalysis generated event stream for video asset A_(i)). In addition, theDMM 66 may receive A_(i) (Video Asset Content being played back) andM_(i)(t) (i.e., a manifest file for A_(i)) from the MDC 64 of thepackager 54. Following the above referenced process, the DMM 66 providesan output to the packager 54 of DM_(i)(t) (i.e., a Dynamic Manifest Mapfor A_(i); reference navigation hints through manifest/content chunksthat are content and usage behavior aware as discussed above). This mayrepresent a custom playlist from mixed bit-rates that the client can useand can be sent along with the manifest files. The packager 54 alsoreceives the adaptive bit rate (ABR) content of the video asset andforwards DM′_(i)(t) (i.e., a manifest file with the associatedDM_(i)(t)) to the client(s) 36.

A flow diagram of method 80 corresponding to the above embodiment isprovided in FIG. 7. In step 82, a manifest for a video asset thatexpresses a set of representations available for each chunk is received.The video asset is of a type that is deliverable in a sequence of chunksto a client device for playback, with each chunk being available in aplurality of different representations and with each representationproviding a different pre-determined level of video quality due todifferent bitrate or video resolution encoding.

For each chunk, data of historic viewing behavior of previous viewersand data of content analysis information of the chunk is received instep 84. As discussed above, the data of historic viewing behaviorprovides an indication of a level of viewer interest and the data ofcontent analysis information provides an indication of a level ofimportance, relevance, or activity of the chunk. Also for each chunk, arepresentation is individually selected in step 86 for the chunk fromthe manifest based on the data of historic viewing behavior and the dataof content analysis information of the chunk.

Thereafter, in step 88, a dynamic manifest for the video asset isgenerated in which the representations individually selected asdescribed above for each chunk are recommended for streaming to theclient device in which different levels of video quality are recommendedfor different chunks.

Dynamic Manifest Mapping Optimizing for Content Delivery NetworkResources

According to another embodiment, information about network resourceutilization may be collected across a set of output transport streams byinstrumented network delivery resources. The type of informationcollected may include {Curr, Ave, Peak, Min} corresponding to {Current,Average, Peak, Minimum} data rate, which is indicative of utilization ofnetwork resources. For Edge QAMs (such as APEX 1000 manufactured byArris Group, Inc.), data rate utilization information is collected.Similar information can be gathered from appropriately instrumentednetwork resource managers that oversee the delivery of IP bit streams.

In an IP delivery system, the above referenced information may be usedto adaptively change or recommend the video manifest sequence thatactively mitigates server-side network congestion. According toembodiments disclosed herein, a network aware manifest manager (NMM)actively manages the recommended delivery of video chunks.

By way of example, a network resource manager may receive informationconcerning network activity across a number of active playback sessions,network bandwidth, and network storage. In addition, a number ofmanifests may be input into the network resource manager. From theseinputs, the network resource manager may create a new set of manifestsbased on the information provided concerning network utilization, userbehavior, content semantics across multiple streams and thereby producemore efficient delivery across multiple streams by more effective usageof network resources.

By way of example, each video asset (A_(i)) may have severalrepresentations, each at a different video bitrate and/or resolution.For instance, each asset (A_(i)) may have (n) representations. Eachrepresentation may have an affiliated bitrate b_(j), j∈[1, n]. Thebitrates are ordered b₁< . . . <b_(n) and lower bitrates generallycorrespond to lower fidelity video quality due to the lower videoencoding bitrate and/or lower video resolution and higher bitratesgenerally correspond to higher fidelity video quality due to the highervideo encoding bitrate and/or higher video resolution.

According to an embodiment, discrete network utilization functions(N_(i)(t)) are produced. For instance, for a time range t∈[t₁,t₂] whichmaps onto a single content chunk c, ∀t∈[t₁,t₂], (Θ(t)=c), the discretenetwork utilization function (N_(i)(t)) is mapped onto one of thediscrete bitrate values or functions [1, n]. This creates a recommendedtrace, NM_(i)(t) through either M_(i)(t) or DM_(i)(t), discussed above,that is aware of network resource utilization. If the NMM is aware ofhigh network resource utilization (e.g., there is currently highthroughput and/or limited additional network capacity), then the valueof NM_(i)(t) may be lower than the value of DM_(i)(t). This would havethe effect of indicating to the client to use a representation with alower bitrate for the chunk at time t. On the other hand, if there issignificant available network capacity, NM_(i)(t) may take a valuehigher than DM_(i)(t). At other times, NM_(i)(t) may be the same valueas DM_(i)(t).

In some embodiments, it may be preferable to limit the rate of change ofthe bitrate from one chunk to the next, because a significant bitratechange may be visually jarring. By way of example, the change rate maybe limited to one step in bitrate per chunk.

For purposes of communicating dynamic manifest information to clients,as the dynamic manifest recommended bitrate is determined for each chunkof a video asset, the recommended bitrate value is communicated to theclient, that is, the device that is receiving the video chunks andplaying the video for the user. This may be accomplished in the samemanner as previously described.

FIG. 5 provides an example of a system 70 in which a Network-awareManifest Manager (NMM) 72 is utilized in association with a DynamicManifest Mapper (DMM) 66 and packager 54 discussed above. The DMM 66creates an array of manifest maps, DM_(i)(t) where 1≤i≤N and N=number ofactive playback sessions in which the suggested manifest playback isbased on user behavior and media content activity (i.e., V_(i)(t) andE_(i)(t)).

The NMM 72 modulates the suggested manifest playback based on contentdelivery network activity. For example, the NMM 72 may receive as aninput: the DM_(i)(t) (i.e., a Dynamic Manifest Map for video asset A_(i)as discussed above) from the DMM 66; and network utilization information(Curr, Ave, Peak, Min) such as from a network resource manager or likeequipment. As an alternative, if a DMM 66 is not utilized or if userbehavior and media content activity (i.e., V_(i)(t) and E_(i)(t)) arenot known, the NMM 72 may receive a regular manifest, M_(i)(t), that hasnot been subject to adjustment based on user behavior and mediasignificance.

With the above information, a Network Modified manifest map NM_(i)(t) isgenerated and output by the NMM 72 over a specified time window(representing some number of chunks). The packager 54 sends NM′_(i)(t)(which is the Manifest file with the associated NM_(i)(t)) to the clientdevice(s) 36.

A flow diagram of method 90 corresponding to the above embodiment isprovided in FIG. 8. In step 92, a manifest for a video asset isreceived. The video asset is of a type that is deliverable in a sequenceof chunks to a client device for playback, with each chunk beingavailable in a plurality of different representations and with eachrepresentation providing a different pre-determined level of videoquality due to different bitrate or video resolution encoding.

For each chunk, data of a level of available network bandwidth, a levelof available network storage, and/or data rate utilization informationof network resources including at least one of current, average, peak,and minimum data rate of network resources is received in step 94. Alsofor each chunk, a representation is individually selected in step 96 forthe chunk from the manifest based on the data of a level of availablenetwork bandwidth, a level of available network storage, and/or datarate utilization information of network resources.

Thereafter, in step 98, a dynamic manifest for the video asset isgenerated in which the representations individually selected asdescribed above for each chunk are recommended for streaming to theclient device in which different levels of video quality are recommendedfor different chunks.

Home Resource Manager

According to a further embodiment, a home or local gateway acts as proxyfor receiving and disseminating IP streams to a plethora of clients atthe client or home or local location. Accordingly, the functionalitydiscussed above may be modified and combined to perform local managementof IP bitstream manifests. Such an embodiment provides the ability tolocally optimize delivery to individual devices in the local homegateway.

By way of example, a so-called “Fusion Service” can be provided with useof a local Dynamic Manifest Mapper (DMM). Equipment providing the fusionservice may receive as an input user behavior related to a contentstream or asset (fusion server), content event stream (temporalmetadata) derived from media analysis, and a manifest file for thecontent. From this information, the equipment may create an activity mapof content/user behavior for streams destined for a home gateway. Alocal network or home resource manager of the home gateway may providecharacterization of in-home or local network activity and an indicationof which manifest bit-rate ranges are best suited for the localenvironment. A local DMM of the home gateway may provide a referencenavigation of manifest chunks that take into account user behavior,content activity, and local network cache and bandwidth usage therebymanaging the bandwidth and storage resources in the home for moreefficient delivery in the home.

As shown in FIG. 6, Fusion Service equipment 74 creates a representationof user interactions/activity and content behavior and outputs anactivity map of content that may be embedded with the manifest files.This may be received by the packager 54 and forwarded to the homegateway 76. The Home Resource Manager (HRM) 78 provides characterizationof in-home activity and an indication of which manifest bit-rate rangesmay be best suited for the local environment. The Local DMM 80 providesa reference navigation of manifest chunks that take into account userbehavior, content activity, and local network cache and bandwidth usage.Accordingly, appropriate adaptive bitrate content is streamed to theclient devices 36.

A system for carrying out any of the above disclosed methods may includesoftware or the like provided on a circuit board or within anotherelectronic device and can include various processors, microprocessors,modules, components, controllers, chips, disk drives, and the like. Itwill be apparent to one of ordinary skill in the art that systems,modules, components, processors, servers, and the like may beimplemented as electronic components, software, hardware or acombination of hardware and software for purposes of providing a system.

Embodiments may also include at least one non-transitory computerreadable storage medium having computer program instructions storedthereon that, when executed by at least one processor, can cause the atleast one processor to perform any of the steps described above.

While the principles of the invention have been described above inconnection with specific devices, apparatus, systems, algorithms, and/ormethods, it is to be clearly understood that this description is madeonly by way of example and not as limitation. One of ordinary skill inthe art will appreciate that various modifications and changes can bemade without departing from the scope of the claims below.

The word “comprise” or a derivative thereof, when used in a claim, isused in a nonexclusive sense that is not intended to exclude thepresence of other elements or steps in a claimed structure or method. Asused in the description herein and throughout the claims that follow,“a”, “an”, and “the” includes plural references unless the contextclearly dictates otherwise. Also, as used in the description herein andthroughout the claims that follow, the meaning of “in” includes “in” and“on” unless the context clearly dictates otherwise.

The above description illustrates various embodiments along withexamples of how aspects of particular embodiments may be implemented,and are presented to illustrate the flexibility and advantages ofparticular embodiments as defined by the following claims, and shouldnot be deemed to be the only embodiments. One of ordinary skill in theart will appreciate that based on the above disclosure and the followingclaims, other arrangements, embodiments, implementations and equivalentsmay be employed without departing from the scope hereof as defined bythe claims. Accordingly, the specification and figures are to beregarded in an illustrative rather than a restrictive sense, and allsuch modifications are intended to be included within the scope of thepresent invention. The benefits, advantages, solutions to problems, andany element(s) that may cause any benefit, advantage, or solution tooccur or become more pronounced are not to be construed as a critical,required, or essential features or elements of any or all the claims.The invention is defined solely by the appended claims including anyamendments made during the pendency of this application and allequivalents of those claims as issued.

We claim:
 1. A method of bitrate adaptation for a video asset to bestreamed to a client device for playback, the video asset beingdeliverable in a sequence of chunks available in a plurality ofdifferent representations with each representation providing a differentpre-determined level of video quality due to different bitrate or videoresolution encoding, comprising the steps of: selecting a representationfrom the plurality of different representations for at least one of thechunks of the video asset from a manifest which expresses a set ofrepresentations available for each of the chunks of the video asset; andgenerating a dynamic manifest for the video asset in which therepresentation selected during said selecting step for the at least oneof the chunks is recommended for streaming to the client device;wherein, during said selecting step, a selection of the representationrecommended for the at least one of the chunks is based on selectioninformation comprising each of: data of historic viewing behavior ofprevious viewers of the at least one of the chunks of the video asset,obtained through collection of user interactions while watching contentand data of content analysis information for the at least one of thechunks of the video asset; and wherein the data of historic viewingbehavior provides an indication of a level of historic viewer interestof each of the chunks of the video asset and the data of contentanalysis information provides an indication of a level of at least oneof importance, relevance, and activity of each of the chunks of thevideo asset.
 2. The method according to claim 1, wherein, during saidselecting step, a representation providing a higher pre-determined levelof video quality is selected for a chunk having historic viewer interestof a pre-determined threshold or having at least one of importance,relevance, and activity of a pre-determined threshold.
 3. The methodaccording to claim 1, wherein said selecting step is repeated for eachof the chunks of the video asset, and wherein the dynamic manifestincludes a recommendation for at least one of the chunks of apre-determined level of video quality that is different from arecommendation of a pre-determined level of video quality recommendedfor at least one other of the chunks of the video asset.
 4. The methodaccording to claim 1, wherein, during said selecting step, the selectioninformation is mapped to one of the pre-determined levels of videoquality.
 5. The method according to claim 1, wherein, during saidselecting step, the representation selected for the at least one of thechunks of the video asset to be streamed to the client devicecorresponds to at least one of a minimum, a maximum, and an average of alevel of the historic viewing behavior and a level of the contentanalysis information.
 6. The method according to claim 1, wherein a rateof change of a level of video quality recommended for any two adjacentchunks of the video asset is limited to a predetermined number of levelsof video quality.
 7. The method according to claim 1, further comprisingthe step of communicating the dynamic manifest to the client device inat least one of a master playlist and an EXT-X-SESSION-DATA tag of HTTPLive Streaming protocol.
 8. The method according to claim 1, furthercomprising the steps of: modifying the representation recommended for atleast one of the chunks of the video asset to be streamed to the clientdevice as selected during said selecting step based on at least one of alevel of available network bandwidth, a level of available networkstorage capacity, and data rate utilization information of networkresources including at least one of current, average, peak, and minimumdata rate of network resources; and generating a modified dynamicmanifest for the video asset such that the representation recommendedand as modified during said modifying step is recommended for the atleast one of the chunks.
 9. The method according to claim 8, wherein,during said modifying step, a representation providing a higherpre-determined level of video quality is selected for a chunk having atleast one of available network bandwidth, available network storage, andcurrent, average, peak, or minimum data rate of a pre-determinedthreshold.
 10. A system of bitrate adaptation for a video asset to bestreamed to a client device for playback, the video asset beingdeliverable in a sequence of chunks available in a plurality ofdifferent representations with each representation providing a differentpre-determined level of video quality due to different bitrate or videoresolution encoding, said system comprising electronic apparatus havingat least one processor configured to: select a representation from theplurality of different representations for at least one of the chunks ofthe video asset from an existing manifest which expresses a set ofrepresentations available for each chunk of the video asset; andgenerate a dynamic manifest for the video asset in which therepresentation selected for the at least one of the chunks isrecommended for streaming to the client device; where the electronicapparatus is a dynamic manifest manipulator, wherein a selection of therepresentation recommended for the at least one of the chunks is basedon each of data of historic viewing behavior of previous viewers of theat least one of the chunks of the video asset and data of contentanalysis information for the at least one of the chunks of the videoasset, and wherein the data of historic viewing behavior provides anindication of a level of historic viewer interest of each chunk of thevideo asset and the data of content analysis information provides anindication of a level of at least one of importance, relevance, andactivity of each chunk of the video asset.
 11. The system according toclaim 10, wherein said at least one processor being configured to selecta representation providing a higher pre-determined level of videoquality for a chunk having historic viewer interest of a pre-determinedthreshold or having at least one of importance, relevance, and activityof a pre-determined threshold.
 12. The system according to claim 10,wherein the electronic apparatus is a network-aware manifest manager,wherein a selection of the representation recommended for the at leastone of the chunks is based on at least one of a level of availablenetwork bandwidth, a level of available network storage, and data rateutilization information of network resources including at least one ofcurrent, average, peak, and minimum data rate of network resources. 13.The system according to claim 12, wherein said at least one processorbeing configured to select a representation providing a higherpre-determined level of video quality for a chunk having at least one ofthe level of available network bandwidth, the level of available networkstorage, and current, average, peak, or minimum data rate of apre-determined threshold.